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Abstract 

Common assumptions on the source producing the words inserted in a suffix trie with n 
leaves lead to a log n height and saturation level. We provide an example of a suffix trie 
whose height increases faster than a power of n and another one whose saturation level 
is negligible with respect to log n. Both are built from VLMC (Variable Length Markov 
Chain) probabilistic sources and are easily extended to families of tries having the same 
properties. The first example corresponds to a "logarithmic infinite comb" and enjoys 
a non uniform polynomial mixing. The second one corresponds to a "factorial infinite 
comb" for which mixing is uniform and exponential. 
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1 Introduction 

Trie (abbreviation of retneval) is a natural data structure, efficient for searching 
words in a given set and used in many algorithms as data compression, spell 
checking or IP addresses lookup. A trie is a digital tree in which words are 
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inserted in external nodes. The trie process grows up by successively inserting 
words according to their prefixes. A precise definition will be given in Section HTTl 
As soon as a set of words is given, the way they are inserted in the trie is de- 
terministic. Nevertheless, a trie becomes random when the words are randomly 
drawn: each word is produced by a probabilistic source and n words are chosen 
(usually independently) to be inserted in a trie. A suffix trie is a trie built on 
the suffixes of one infinite word. The randomness then comes from the source 
producing such an infinite word and the successive words inserted in the tree are 
far from being independent, they are strongly correlated. 

As a principal application of suffix tries one can cite the lossless compression 
algorithm Lempel-Ziv 77 (LZ77). The first results on the average size of suffix 
tries when the infinite word is given by a symmetrical memoryless source are due 
to Blumer et al. [l| and those on the height of the tree to Devroye [3]. Using 
analytic combinatorics, Fayolle [gI has obtained the average size and total path 
length of the tree for a binary word issued from a memoryless source (with some 
restriction on the probability of each letter). 

Here we are interested in the height i/„ and the saturation level In of a suffix 
trie Tn containing the first n suffixes of an infinite word produced by a source 
associated with a so-called Variable Length Markov Chain (VLMC) (see Rissanen 



ll| for the seminal work, Galves-Locherbach [8j] for an overview, and [2| for 
a probabilistic frame). One deals with a particular VLMC source associated 
with an infinite comb, described hereafter. This particular model has the double 
advantage to go beyond the cases of memoryless or Markov sources and to provide 
concrete computable properties. The analysis of the height and the saturation 
level is usually motivated by optimization of the memory cost. Height is clearly 
relevant to this point; saturation level is algorithmically relevant as well because 
internal nodes below the saturation level are often replaced by a less expansive 
table. 

All the tries or suffix tries considered so far in the literature have a height and a 
saturation level both growing logarithmically with the number of words inserted, 
to the best of our knowledge. For plain tries, when the inserted words are in- 
dependent, the results due to Pittel 13] rely on two assumptions on the source 
producing the words: first, the source is uniformly mixing, second, the probability 
of any word decays exponentially with its length. Let us also mention the general 
analysis of tries by Clement-Flajolet-Vallee [3| for dynamical sources. For suffix 
tries, Szpankowski jl2| obtains the same result, with a weaker mixing assumption 
(still uniform though) and the same hypothesis on the measure of the words. 
Our aim is to exhibit two cases when these behaviours are no longer the same. 
The first example is the ^^logarithmic comb" , for which we show that the mixing 
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is slow in some sense, namely non uniformly polynomial (see Section 13.21 for a 
precise statement) and the measure of some increasing sequence of words decays 
polynomially. We prove in Theorem 14 . 8 1 1 hat the height of this trie is larger than 
a power of n (when n is the number of inserted suffixes in the tree). The second 
example is the factorial comb" , which has a uniformly exponential mixing, thus 



fulfilling the mixing hypothesis of Szpankowski [12[, but the measure of some 
increasing sequence of words decays faster than any exponential. In this case we 
prove in Theorem 14.91 that the saturation level is negligible with respect to log n. 

We prove more precisely that, almost surely, in ^ o ^ (iog°o^„)^i j , for any S > I. 

The paper is organised as follows. In Section [2l we define a VLMC source associ- 
ated with an infinite comb. In Section, [3] we give results on the mixing properties 
of these sources by explicitely computing the suitable generating functions in 
terms of the source data. In Section HJ the associated suffix tries are built, and 
the two uncommon behaviours are stated and shown. The methods are based on 
two key tools concerning pattern return time: a duality property and the compu- 
tation of generating functions. The relation between the mixing of the source and 
the asymptotic behaviour of the trie is highlighted by the proof of Proposition 14. 71 



2 Infinite combs as sources 

In this section, a VLMC probabilistic source associated with an infinite comb 
is defined. Moreover, we introduce the two examples given in introduction: the 
logarithmic and the factorial combs. We begin with the definition of a general 
variable length Markov Chain associated with a probabilized infinite comb. 
The following presentation comes from Let A be the alphabet {0, 1} and 
jC = A^^ be the set of left-infinite words. Consider the binary tree (represented in 
Figure [1]) whose finite leaves are the words 1, 01, . . . , O'^l, . . . and with an infinite 
leaf 0°° as well. Each leaf is labelled with a Bernoulli distribution, respectively 
denoted hj qQki, k ^ and go°°- This probabilized tree is called the infinite comb. 
The VLMC (Variable Length Markov Chain) associated with an infinite comb is 
the ^-valued Markov chain (V^)„^o defined by the transitions 

P(K+i = Vna\Vn) = g^f (y„)(a) 

where a G ^ is any letter and pref (Ki) denotes the first suffix of Vn (reading from 
right to left) appearing as a leaf of the infinite comb. For instance, if 14 = • • • 1000, 
then pref (Vn) = 0001. Notice that the VLMC is entirely determined by the data 
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Figure 1: An infinite comb 
go°°, 'Zo'-ii ^ ^ 0. From now on, denote Cq = 1 and for n ^ 1, 

n-l 

Cn := n^on(O). 



It is proved in [i*] tliat in tlie irreducible case i.e. wlien go°°(0) 7^ 1, tliere exists a 
unique stationary probability measure vr on £ for (V^)„ if and only if the series 
^ Cn converges. From now on, we assume that this condition is fulfilled and we 
call 

5(x):=^c„x" (1) 

its generating function so that S{1) = J2n^o ^^'^ ^'^y finite word w, we denote 
'k{w) := 7i{Cw). Computations performed in [2| show that for any n ^ 0, 

7r(10") = ^ and ^{0^) = ^^. (2) 

Notice that, by stationarity 7r(0") = 7r(0"'^"'^) + 7r(10"') and by disjointness of 
events, 7r(0") = ^(0"+^) + 7r(0"l) for all n ^ 1 so that 

^(10") = ^(0"1). (3) 
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If Un denotes the final letter of Vn, the random sequence W = U0U1U2 ... is a 
right-infinite random word. We define in this way a probabilistic source in the 
sense of information theory i.e. a mechanism that produces random words. This 
VLMC probabilistic source is characterized by: 

Pu, := F{W has w as a prefix ) = n{w), 

for every finite word w. Both particular suffix tries the article deals with are built 
from such sources, defined by the following data. 



Example 1: the logarithmic comb 

The logarithmic comb is defined by Cq = 1 and for n ^ 1, 

1 



n{n + l){n + 2){n + 3) 
The corresponding conditional probabilities on the leaves of the tree are 

1 4 

gi(0) = — and for 1, go"i(0) = 1 — 



24 - , -.u XV , 

The expression of c„ was chosen to make the computations as simple as possible 
and also because the square-integrability of the waiting time of some pattern will 
be needed (see end of Section H73|) . guaranteed by 



n^Cn < +00. 



Example 2: the factorial comb 

The conditional probabilities on the leaves are defined by 

9o"i(0) = forn ^ 0, 

n + 2 

so that 

1 
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3 Mixing properties of infinite combs 

In this section, we first precise what we mean by mixing properties of a random 
sequence. We refer to Doukhan [5|, especially for the notion of -0- mixing defined 
in that book. We state in Proposition 13.21 a general result that provides the 
mixing coefficient for an infinite comb defined by {Cn)n^o or equivalently by its 
generating function S. This result is then applied to our two examples. The 
mixing of the logarithmic comb is polynomial but not uniform, it is a very weak 
mixing; the mixing of the factorial comb is uniform and exponential, it is a very 
strong mixing. Notice that mixing properties of some infinite combs have already 
been investigated by Isola jol, although with a slight different language. 



3.1 Mixing properties of general infinite combs 

For a stationary sequence (t/n)n^o with stationary measure tt, we want to measure 
by means of a suitable coefficient the independence between two words A and B 
separated by n letters. The sequence is said to be "mixing" when this coefficient 
vanishes when n goes to +oo. Among all types of mixing, we focus on one of the 
strongest type: ip-mixing. More precisely, for ^ m ^ +oo, denote by J-"™ the 
(T-algebra generated by {f/fe, ^ A; ^ m} and introduce for A G J^l^ and B G 
the mixing coefficient 



^{n,A,B) 



{A n r-(™+i)-"S) - n{A)n{B) 



n{A)n{B) 

A)n(B) 

(4) 



^^^^^^TTiAwB) - n{A)n{B 



Tr{A)TT{B) 

where T is the shift map and where the sum runs over the finite words w with 
length \w\ = n. 

A sequence {Un)n^o is called ip-mixing whenever 

lim sup |'0(r;,, y4, 5)1 = 0. 

In this definition, the convergence to zero is uniform over all words A and B. This 
is not going to be the case in our first example. As in Isola [oj, we widely use the 
renewal properties of infinite combs (see Lemma 13.11) but more detailed results 
are needed, in particular we investigate the lack of uniformity for the logarithmic 
comb. 
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Notations and Generating functions 

• For a comb, recall that S is the generating function of the nonincreasing se- 
quence {cn)n^o defined by ([1]). 

• Set po = and for n ^ 1, 

Pn ■ C-n—l Cfiy 

with generating function 

P{x) := 

• Define the sequence (iin)„^o by = 1 and for n ^ 1, 

^ ' ^ ' \w\=n-l 



and let 

denote its generating function. Hereunder is stated a key lemma that will be 
widely used in Proposition 13.21 In some sense, this kind of relation (sometimes 
called Renewal Equation) refiects the renewal properties of the infinite comb. 

Lemma 3.1 The sequences {un)n^o one? (pn)n^o (ii"^ connected by the relations: 

Vn ^ 1, Un = Pn + UiPn-l + ■ ■ ■ + Un-lPl 

and (consequently) 

UnX - — 

n=0 



U{x) = V^/nX" = — - = — — , 

t^n 1-P{X) {1-X)S{X) 



Proof. For a finite word w = ai . . . am such that w ^ 0™, let l{w) denote the 
position of the last 1 in w, that is l{w) := max{l < i < m, = 1}. Then, the 
sum in the expression ([5]) of m„ can be decomposed as follows: 



n-l 



\w\=n—l i=l \w\^n-l 

l(w) — i 

Now, by disjoint union 7r(10"-i) = 7r(10"-il) + 7r(10"), so that 
7r(10"-4) = 7r(l)(c„_i - c„) = 7r(l)p„. 
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In the same way, ior w — ai . . . ctn-i, if l{w) — i then 7r(lt(;l) = 7r(lQ;i . . . Q;j_il)p„_j, 
so that 



i=l ^ ^ \w\=i-l 

n-1 

— Pn ~l~ ^ ^ pn—iUii 



1=1 

which leads to ?7(a;) = (1 — P{x))~^ by summation. □ 
Mixing coefficients 

The mixing coefficients ■0(n, A, S) are expressed as the n-th coefficient in the 
series expansion of an analytic function M"^'^ which is given in terms of S and U . 
The notation means the coefficient of in the power expansion of A[x) 

at the origin. Denote the remainders associated with the series S{x) by 

r„ := ^ Cfe, Rn{x) := ^ c^x^ 

k^n k^n 

and for a ^ 0, define the "shifted" generating function 

1 . X — 1 

Pa{x) := —Y^ pa+nX"^ = X -\ -Ra+l{x). (6) 

Proposition 3.2 For ani/ finite word A and any word B, the identity 

ij{n,A,B) = [x"+^]M^'^(x) 
holds for the generating functions M^'^ respectively defined by: 
i) if A = A'l and B = IB' where A' and B' are any finite words, then 

rAB,^. S{X)-S{1)^ 



[x - l)S{x) 



ii) if A — A' 10"- and B — 0^1 B' where A' and B' are any finite words and 
a + b'^ 1, then 

M^'^'ix) := S{l)^Pa+b{x) + U{x) [S{l)Pa{x)Ph{x) - S{x)] ; 

CaCb 
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Hi) if A = 0'^ and B = & with a,b ^ 1, then 



Tan 



Sil)Raix)Rbix) 



TaHX' 



a+b-2 



-S{x) 



iv) if A = A'lO" and B = & where A' is any finite words and a,b ^ 0, then 



S{l)Pa{x)R,{x) 

T—. b(x) 

Canx" ^ 



v) if A = 0°- and B = 0''1B' where B' is any finite words and a,b ^ 0, then 



M^'^ix) := 8(1) + Uix) 



TaCbX 



S{l)Ra{x)P,{x) 



TaCbX' 



a-1 



S{x) 



Remark 3.3 R is worth noticing that the asymptotics of i/jln, A, B) may not be 
uniform in all words A and B. We call this kind of system non-uniformly ip- 
mixing. R may happen that i^in, A, B) goes to zero for any fixed A and B, hut 
(for example, in case Hi)) the larger a orb, the slower the convergence, preventing 
it from being uniform. 



Proof. The following identity has been established in |2| (see formula (17) in 
that paper) and will be used many times in the sequel. For any two finite words 
w and w' , 

ti:{w1w')ti{\) = 'k{w1)'k{1w'). (7) 
i) li A = A'l and B=1B', then © yields 

n{AwB) = 7c{A'lwlB') = ^^i^7r{lwlB') = S{l)n{A)7i{B)^^^'^'^^ 



vr(l) 



So 



^{n,A,B) = S{l)ur,+i-l 
and by Lemma [3. 11 the result follows. 

ii) Let A = A'W and B = 0''1B' with a,b ^ and a + b ^ 0. To begin with, 



7r{AwB) = ^7r(A'l)^(10"it;0Hfi'^ 



TT 



(1) 



TT 



1 



;7i{A'l)n{10''w0h)n{lB'). 
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Furthermore, ti{A) = Ca7r(y4'l) and by ([3]), 7r(0''l) = 7r(10''), so it comes 



n{B) = -^MOhMlB') = ^7r(lS') = c,n{lB'). 
7r(l) 7r(l) 



Therefore, 



CaCb7T{iy 



Using 7r(l)S'(l) = 1, this proves 



,^in,A,B) = S{l)^" 



CaCb 

where 



|ui|=?i— 1 



As in the proof of the previous lemma, if w = ai . . . am is any finite word 
different from O'", we call f{w) := min{l < i < m,aj = 1} the first place 
where 1 can be seen in w and recall that l{w) denotes the last place where 
1 can be seen in w. One has 

J2 vr(10"wO''l) = 7r(10"+""^+H) + ^ ^ 7r(10"u)0''l). 

|?i)|=n— 1 l<*<i<" — 1 |™| = n-i 

U i = j then w is the word 0*~^10"~*~^, else w is of the form 0*~^liy'10"~^~-', 
with \w'\ = j — i — 1. Hence, the previous sum can be rewritten as 

7r(10"u;0^1) = Tr{l)pa+b+n + Tr{l)Ypa+ipn-i+b 

\w\=n—l i=l 

+ Y Yl 7r(10'^+*-il«;10"-^--'+^l). 

Equation ([7j) shows 

= Pa+iPn-j+bT^iM). 
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This implies: 

~ Pa+b+n + ^ ] Pa+iPn-i+b + ^ ] Pa+iPn-j+b ^ ] 



" ^ Trfltol) 



7r(l) 

Recalling that uq = 1, one gets 

^ ~ Pa+b+n + ^ ] Pa+iPn-j+bUj~i 
l<i<j<n-l 

which gives the result ii ) with Lemma 13.11 
Hi) Let A = 0" and S = O'' with a,b^l. Set 

^ ^ \w\=n-l 

First, recall that, due to ([2]), vr(v4) = 7r(l)ra and n^B) = 7r(l)rb. Conse- 
quently, 

Let w he a. finite word with \w\ = n — 1. If w = 0"~^, then 

7r(A«;S) = 7r(0"+"-i+^) = 7r(l)r,+fe+„_i. 

If not, let f{w) denote as before the first position of 1 in zi; and l{w) the 
last one in w. If f{w) = l{w), then 

= ^7r(0'^+/("')-ll)7r(10"-l-^('")+'') = n{l)Ca+fi^)^lCn^l-fiy,)+b. 

If f{w) < l{w), then writing w = Wi . . . Wn-i, 

niAwB) = 7r(0'^+/("')-ilwy(^)+i . . . u;,(^)_ilO"-i-'('")) 

= ^7^(0"+^^")-'l)vr(l«;/(.„)+i . . . «;,(^)„il)7r(10"-i-'("')+''). 

Summing yields 

ya,b _ + ^ Cg^i-iCn-l+b-i + ^ ] ^ ] C(j+j_i j;— — C„_i_|_(,__ 

= 1^a+b+n-l + ^ ] Ca+i-lCn-l+b-jUj^i, 
l<i<j<n-l 

which gives the desired result. The last two items, left to the reader, follow 
the same guidehnes. □ 
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3.2 Mixing of the logarithmic infinite comb 

Consider the first example in Section |2], that is the probabihzed infinite comb 
defined by cq = 1 and for any n ^ 1 by 



" n(n + l)(n + 2)(n + 3)' 

When |x| < 1, the series S{x) writes as follows 

._ 47 5 1 (l-x)^log(l-x) 
36" 12^ + 6^+ 6^^ 

and 

19 



With Proposition I3.2[ the asymptotics of the mixing coefficient comes from sin- 
gularity analysis of the generating functions M"^'^. 

Proposition 3.4 The VLMC defined by the logarithmic infinite comb has a non- 
uniform polynomial mixing of the following form: for any finite words A and B, 
there exists a positive constant Ca,b such that for any 1, 

Remark 3.5 The Ca,b cannot be bounded above by some constant that does not 
depend on A and B, as can be seen hereunder in the proof. Indeed, we show that 
if a and b are positive integers, 



1/;(r^,0^0 



1 fSjl) 1 1 ^ 1 \ 1 

raTb Ta Tfe S{1) ) 



as n goes to infinity. In particular, -0(71,0,0") tends to the positive constant ^. 
Proof of Proposition 13.41 

For any finite words A and B in case i) of Proposition [221 one deals with U {x) = 
((1 — x)S{x))~'^ which has 1 as a unique dominant singularity. Indeed, 1 is the 
unique dominant singularity of S, so that the dominant singularities of U are 1 
or zeroes of S contained in the closed unit disc. But S does not vanish on the 
closed unit disc, because for any z such that \z\ ^ 1, 

I^WI»'-S: „(„+l)(„V2)(n + 3) =^-'"W-^>-II- 
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Since 



M{x) = f^\^f^^\^ = Sil)U{x) 



1 



{x — l)S{x) ' ' ' ' 1 — x' 

the unique dominant singularity of M is 1, and when x tends to 1 in the unit 
disc, ([8]) leads to 

M{x) = A{x) - ^^(1 - xY log(l -x) + 0{{l- xf log(l - x)) 

where A{x) is a polynomial of degree 2. Using the classical transfer theorem (see 
Flajolet and Sedgewick 0, section VI]) based on the analysis of the singularities 
of M, we get 



tp{n-l,wl,lw') = [x"]M(x) 



11 /I 

+ 



35(1)^3 

The cases ii)^ iii), iv) and v) of Proposition 13.21 are of the same kind, and we 
completely deal with case iii). 

Case iii): words of the form A = 0°" and B = 0^, a,b ^ 1. As shown in 
Proposition 13. 2[ one has to compute the asymptotics of the n-th coefficient of the 
Taylor series of the function 



M-^\x) := 5(l)^^r,+6+„_ix" + U{x) 



ran 



S{l)K{x)Rb{x) 



-S{x) 



(9) 



The contribution of the left-hand term of this sum is directly given by the asymp- 
totics of the remainder 



3n(n + l)(n + 2) 3^3 



1 / 1 

+ 



By means of singularity analysis, we deal with the right-hand term 



N'^'^x) := U{x) 



S{l)K{x)Rt{x) 



r^nx' 



■a+b-2 



S{x) 



Since 1 is the only dominant singularity of S and U and consequently of any Ra, 
it suffices to compute an expansion of N^^^x) at a; = 1. It follows from 1^ that 
U, S and Ra admit expansions near 1 of the forms 

1 , . , 1 



U{x) 



sim-x) 



+ polynomial + 



65(1)^ 



^l-a;)^log(l-x) + C(l-x) 



S(x) = polynomial H — (1 — a;)'^log(l — x) + 0{1 — x)^, 

6 
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and 

Ra(x) = polynomial + -(1 - x)^log(l - x) + 0(1 - x)^. 

6 

Consequently, 

in a neighbourhood of 1 in the unit disc so that, by singularity analysis, 
[x-]N^\x) = -U- + --4u)^ + 4-s 



Consequently ([9]) leads to 



^a c^b^ _ ^n...a,b(^^ ^ ( S{1) 1 1 , 1 1 



as n tends to infinity, showing the mixing inequality and the non uniformity. 
The remaining cases ii), iv) and v) are of the same fiavour. □ 

3.3 Mixing of the factorial infinite comb 

Consider now the second Example in Section |2l that is the probabilized infinite 
comb defined by 

Vn G N, Cn= , , ... . 

[n + 1)! 

With previous notations, one gets 

S{x) = and U{x 



X ' ' (1 - a;)(e^ - 1)' 

Proposition 3.6 The VLMC defined by the factorial infinite comb has a uniform 
exponential mixing of the following form: there exists a positive constant C such 
that for any n ^ 1 and for any finite words A and B, 

\^{n,A,B)\<: ^ 



(27r)" 
Proof. 
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i) First case of mixing in Proposition 13.21 A = A'l and B = IB'. 



Because of Proposition I3.2[ the proof consists in computing the asymp- 
totics of [x"']M{x). We make use of singularity analysis. The dominant 
singularities of 

^' (x-i)S(x) 
are readily seen to be 2i7r and — 2i7r, and 

1 -e 1 



M{x) 



2i7v 1 - 2iTT 1 - tS- 

2tTT 



The behaviour of M in a neighbourhood of —2in is obtained by complex 
conjugacy. Singularity analysis via transfer theorem provides thus that 



2(e- 1) / 1 ^ " 



where 

1 if n is even 



271 if n is odd. 

ii) Second case of mixing: A = A'lO"' and B = O^IB'. 

Because of Proposition 13.21 one has to compute [x"]M°''^{x) with 

M^\x) := S{l)^Pa+>,ix) + -i- ■ \s{l)Paix)P,{x) - S{x) 

CaCb b[X)l — XL 

where Pa+b is an entire function. In this last formula, the brackets contain 
an entire function that vanishes at 1 so that the dominant singularities of 
M*^'^ are again those of S~^, namely ±2in. The expansion of M"'^(x) at 
2z7r writes thus 

Ma,b^^) -Sil)Pai2in)ni2z7r) 1 



2iTT 1 - 2iTV I - 7p- 

2117 

which implies, by singularity analysis, that 

Besides, the remainder of the exponential series satisfies 



Ex"- X f X , ^ 

^ = ^ 1 + - + 10 
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when a tends to infinity. Consequently, by Formula (j6]), Pa(2z7r) tends to 
2m as a tends to infinity so that one gets a positive constant Ci that does 
not depend on a and h such that for any n ^ 1, 

\ij{n,A,B)\^ 



Hi) Third case of mixing: ^4 = 0^^ and B = O''. 

This time, one has to compute [x"]M"'''(x) with 

'Sil)Raix)Rb{x) 

7a+b+n-lX -1- U yx) 



-S{x) 



the first term being an entire function. Here again, the dominant singular- 
ities of M'^'^ are located at ±2z7r and 

^a,.. ^ ^ -S{l)Ra{2l1l)R,{2t'K) 1 



2m (1 - 2i'K)ran{2mY+^-^ 1 - ^ 
which implies, by singularity analysis, that 

1-e Ra{2i7i)Rb{2i7v) 1 



ilj{n-l,A,B) ~ 23? 



oo \l-2i7r raH{2mY+^~'^ (2«7r)' 

Once more, because of (fTOj) . this implies that there is a positive constant 
C2 independent of a and h and such that for any n ^ 1, 

\ij{n,A,B)\^ 



(2vr) 



iv) and &ot/i remaining cases of mixing that respectively correspond to words 
of the form A = A'W, B = & and A = 0", B = O^IB' are of the same 
vein and lead to similar results. O 



4 Height and saturation level of suffix tries 

In this section, we consider a suffix trie process (7^)„ associated with an infinite 
random word generated by an infinite comb. A precise definition of tries and suffix 
tries is given in section 14.11 We are interested in the height and the saturation 
level of such a suffix trie. 
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Our method to study these two parameters uses a duahty property a la Pittel 
developed in Section \A.2\ together with a careful and explicit calculation of the 
generating function of the second occurrence of a word (in Section I4l3|) which can 
be achieved for any infinite comb. These calculations are not so intricate because 
they are strongly related to the mixing coefficient and the mixing properties 
detailed in Section |3l 

More specifically, we look at our two favourite examples, the logarithmic comb 
and the factorial comb. We prove in Section that the height of the first one is 
not logarithmic but polynomial and in Section H^l that the saturation level of the 
second one is not logarithmic either but negligibly smaller. Remark that despite 
the very particular form of the comb in the wide family of variable length Markov 
models, the comb sources provide a spectrum of asymptotic behaviours for the 
suffix tries. 

4.1 Suffix tries 

Let (3^n)n>i be an increasing sequence of sets. Each set 3^„ contains exactly 
n infinite words. A trie process (7^)„>i is a planar tree increasing process 
associated with (3^„)n>i- The trie Tn contains the words of 3^^ in its leaves. It 
is obtained by a sequential construction, inserting the words of 3^ri successively. 
At the beginning, 7i is the tree containing the root and the leaf . . . (resp. the 
leaf 1 . . . ) if the word in J^i begins with (resp. with 1). For n > 2, knowing the 
tree Tn-i, the n-th word m is inserted as follows. We go through the tree along 
the branch whose nodes are encoded by the successive prefixes of m; when the 
branch ends, if an internal node is reached, then the word is inserted at the free 
leaf, else we make the branch grow comparing the next letters of both words until 
they can be inserted in two different leaves. As one can clearly see on Figure [2] a 
trie is not a complete tree and the insertion of a word can make a branch grow by 
more than one level. Notice that an internal node exists within the trie if there 
are at least two words in the set starting by the prefix associated to this node. 
This indicates why the second occurrence of a word is prominent. 

Let m := 010203 ... be an infinite word on ^ = {0, 1}. The sufRx trie %i (with 
n leaves) associated with m, is the trie built from the set of the n-th first suffixes 
of m, that is 

yn = {m, 02O3 . . . , 03O4 a„o„+i . . .}. 

For a given trie Tn, we are mainly interested in the height Hn which is the maximal 
depth of an internal node of Tn and the saturation level In which is the maximal 
depth up to which all the internal nodes are present in Tn- Formally, if dTn 



17 



Cenac, Chauvin, Paccaut, Pouyanne: uncommon suffix tries 





Figure 2: Last steps of the construction of a trie built from the set 
(000 . . . , 10 . . . , 1101 . . . , 001, OHIO 1100 ... , 01111 . . .). 



denotes the set of leaves of 7^, 
Hn = 

L = max {j G N| #{w G % \ 8%, 
See Figure O for an example. 



max { I I } 

ueTn\dTn 



U\ 



4.2 Duality 

Let {Un)n^i be an infinite random word generated by some infinite comb and 
(J~n)n^i be the associated suffix trie process. We denote by 71 the set of right- 
infinite words. Besides, we define hereunder two random variables having a key 
role in the proof of Theorem 14.81 and Theorem 14.91 This method goes back to 



Pittel 10 



Let s G 7^ be a deterministic infinite sequence and s*^*^^ its prefix of length k. For 
n > 1, 



Tkis) 



if s^^-* is not in Tn 

max{A; ^ 1 | the word s^'^^ is already in 7^ \ dTn}, 
min{n ^ 1 | Xn{s) = k}, 
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where "s^'^^ is in 7^ \ 97^" stands for: there exists an internal node v inTn such 
that s'^^^ encodes v. For any k ^ 1, Tk{s) denotes the number of leaves of the first 
tree "containing" s^^\ See Figure H] for an example. Thus, the saturation level 
in and the height Hn can be described using 

£r,=minXn(s) and i7„ = maxX„(s). (11) 

sen sen 

Moreover, X„(s) and Tk{s) are in duality in the following sense: for all positive 
integers k and n, one has the equality of the events 

{X„(s) ^k} = {n{s) ^ n}. (12) 

The random variable Tk{s) (if k ^ 2) also represents the waiting time of the 
second occurrence of the deterministic word s^''^ in the random sequence (f/„)n>i, 
i.e. one has to wait Tk{s) for the source to create a prefix containing exactly two 
occurrences of s'-'^^ More precisely, for k ^ 2, Tk{s) can be rewritten as 

Tk{s) = min|n ^ 1 \UnUn+i ■ ■ ■ Un+k~i = s^''^ and 3!j < n such that 
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Figure 4: Example of suffix trie with n = 20 words. The saturation level is 
reached for any sequence having 1000 as prefix (in red); £20 = -^20 (s) = 3 and 
thus 73(5) ^ 20. The height (related to the maximum of X20) is realized for any 
sequence of the form 110101 .. . (in blue) and H20 = 6. Remark that the shortest 
branch has length 4 whereas the saturation level in is equal to 3. 

Notice that Tk{s) denotes the beginning of the second occurrence of s*^^-* whereas 
in [2I, r^^^ (s*-'^^) denotes the end of the second occurrence of s^''\ so that 

^(2) (^W) ^ ^ ^_ ^^3) 

More generally, in [2] , for any r ^ 1, the random return times t^^\w) is defined 
as the end of the r-th occurrence of w in the sequence (f/n)n>i and the generating 
function of the r(^) is calculated. We go over these calculations in the sequel. 

4.3 Return time generating functions 

Proposition 4.7 Let k ^ 1. Let also w = 10'^"^ and t^'^^w) be the end of the 
second occurrence of w in a sequence generated by a comb defined by (c„)„j.o. 
Let S and U be the ordinary generating functions defined in Section \3.1\ The 
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probability generating function of t^'^\w) is 



Sil){l-x) [l + Cfe_ix'=-i(f/(x)-l)] 



2 • 



Furthermore, as soon as '^n^i'n^Cn < oo, the random variable t^'^\w) is square- 
integrable and 

E(r(2)H) = ^ + of^V Var(r(^)H) = ^^ + of^V (14) 
Proof. For any r ^ 1, let T^''\w) denote the end of the r-th occurrence of w 

(r) 

in a random sequence generated by a comb and $«, its probabihty generating 
function. The reversed word of c = ai. . .on will be denoted by the over line 
c := ajM . . . «! 

We use a result of [2;] that computes these generating functions in terms of sta- 
tionary probabilities qi"'\ These probabilities measure the occurrence of a finite 
word after n steps, conditioned to start from the word c. More precisely, for any 
finite words u and c and for any n ^ 0, let 

gW(ti) := TT (^7„_|„| + |c| + l • • • Un+\c\ = u\Ui . . . ?7|c| = c) . 

It is shown in ^ that, for |a;| < 1, 



«fM(x) = <|.«(x)(l--^~' ' 



and for r > 1, 
where 

oo 
n=k 

k-l 

Cw{x) := 1 + '^{wj+i...Wk=wi...Wk-j}(l^{ (w) {"^k-j+l ■ ■ ■ Wk) . 

In the particular case when w = lO'^"^, then pref (w) = w = O'^^^l and 7t(w) 



Cfc-l 

S(l) 



. Moreover, Definition (jl]) of the mixing coefficient and Proposition 13.21 i) 
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imply successively that 

•^^f (w) (^) = ^ (Un-k-\w\+i ■ ■ ■ Un+k = w Ui . . .Uk = pref (w] 
= 7t{w) (^tp ~ pref (w) ,w) + 1^ 

This relation makes more explicit the link between return times and mixing. This 
leads to 



Furthermore, there is no auto-correlation structure inside w so that Cw{x) = 1 
and 

S^{x) = 1 + Ck-ix'''^{U{x) - l). 

This entails 



X) 



Ck-lX 



and 



S{l){l-x) [l + Ck-ix''-'{U{x)-l)] 



Sw{x) 

^2 ^2k~l 



^x^''~^{U{x) - 1) 



S{l){l-x) [l + Ck-ix'^-'{U{x)-l)]- 
which is the announced result. The assumption 

n^Cn < oo 



makes U twice differentiable and elementary calculations lead to 

(*i^^y(i) = (€^)'(i) + ^, 



Ck-l 



'^k-1 V^fc-l 



5(1)' 

' ^ and ($L^)r(l) = ^ 



1 

2 + ^ \ 2 

'^k-1 V'^fc-l 



and finally to ([T 



n 
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4.4 Logarithmic comb and factorial comb 

Let /i+ and h_ be the constants in [0, +00] defined by 

:= lim — max/in f — - — | and h- := lim — min/lnf — - — |, (15) 
n~>+oo n I V7r(w)/ J n-5>+oo n L \7r[W)/ J 

where the maximum and the minimum range over the words w of length n with 



n (w) > 0. In their papers, Pittel [10| and Szpankowski [12[ only deal with the 
cases < +00 and h_ > 0, which amounts to saying that the probability of 
any word is exponentially decreasing with its length. Here, we focus on our two 
examples for which these assumptions are not fulfilled. More precisely, for the 
logarithmic infinite comb, (|2]) implies that 7r(10") is of order n"^, so that 

1 . / 1 \ , In 77, 
/i_ ^ lim -In — TT = 4 lim = 0. 

n-s>+oo n V7r(10"^^)/ n 

Besides, for the factorial infinite comb, 7r(10") is of order ^^^^^| so that 

1 / 1 \ n\ 
^ lim — ln( — ; — ) = lim — = +00. 

For these two models, the asymptotic behaviour of the lengths of the branches is 
not always logarithmic, as can be seen in the two following theorems, shown in 
Sections 14.51 and 14.61 

Theorem 4.8 (Height of the logarithmic infinite comb) Let Tn he the suf- 
fix trie built from the n first suffixes of a sequence generated by a logarithmic 
infinite comb. Then, the height Hn of Tn satisfies 

V5 > 0, 1 — > +00 in probability. 

Theorem 4.9 (Saturation level of the factorial infinite comb) Let Tn be 

the suffix trie built from the n first suffixes of the sequence generated by a factorial 
infinite comb. Then, the saturation level In ofTn satisfies: for any 5 > 1, almost 
surely, when n tends to infinity. 



tn e 



log 77, 



(log log n) 



The dynamic asymptotics of the height and of the saturation level can be visu- 
alized on Figure El The number n of leaves of the suffix trie is put on the x-axis 
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while heights or saturation levels of tries are put on the y-axis. Plain lines rep- 
resent a logarithmic comb while long dashed lines are those of a factorial comb 
(mean values of 25 simulations). 

Short dashed lines represent a third infinite comb defined by the data c„ = 
1 11^=1 + (i+fc)2 j for n ^ 1. Such a process has a uniform exponential mixing, 



a finite /i+ and a positive h- as can be elementarily checked. As a matter of con- 



sequence, it satisfies all assumptions of Pittel [10[ and Szpankowski [12| implying 
that the height and the saturation level are both of order log n. Such assumptions 
will always be fulfilled as soon as the data {cn)n satisfy lim.ncl/"' < 1; the proof 
of this result is left to the reader. 

One can notice the height of the logarithmic comb that grows as a power of n. 
The saturation level of the factorial comb, negligible with respect to logn is more 
difficult to highlight because of the very slow growth of logarithms. 

These asymptotic behaviours, all coming from the same model, the infinite comb, 
stress its surprising richness. 

4.5 Height for the logarithmic comb 

In this subsection, we prove Theorem 14.81 

Consider the right-infinite sequence s = 10°°. Then, Tk{s) is the second occur- 
rence time of w = 10*^"^. It is a nondecreasing (random) function of k. Moreover, 
Xn{s) is the maximum of all k such that s^''^ G 7^. It is nondecreasing in n. So, 
by definition of Xn{s) and Tk{s), the duality can be written 

\fn,\/uj, 3kn, kn ^ Xn{s) <kn + l and Tfc„(s) < Tfc„+i(s). (16) 

Claim: 

lim Xn{s) = +00 a.s. (17) 

Indeed, if Xn{s) were bounded above, by K say, then take w = 10^ and consider 
Tk+i{s) which is the time of the second occurrence of 10^. The choice of the 
Cn in the definition of the logarithmic comb implies the convergence of the series 
^^n^c„. Thus (HM holds and E[ri^+i(s)] < oo so that Tk+i{s) is almost surely 
finite. This means that for n > Tk+i{s), the word 10^ has been seen twice, 
leading to Xn{s) ^ K + 1 which is a contradiction. 

We make use of the following lemma that is proven hereunder. 
Lemma 4.10 For s = 10°°, 

w n ^'^(^) n • 

> (J, — — — — )■ tn probammy, 

l^^+V k->-oo 
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and 

With notations ( fT6i) . because of ( fTTl) . the sequence (/c„) tends to infinity, so that 
(Tfc„(s)) is a subsequence of (rfc(s)). Thus, ( |T8l) imphes that 

Vr/ > ^, ^ a.s. and Vr^ > 0, ^ A 0. 

Using duahty (ITB]) again leads to 



p 



In otherwords 

V5 > 0, ^ A +00 

flJ^" n— i>oo 

SO that, since the height of the suffix trie is larger than Xn{s) 



V5>0, +00. 

This ends the proof of Theorem I4.8[ □ 

Proof of Lemma [4.101 
Combining ( IT3l) and ( IT4l) shows that 



p 



E(Tfc(s)) = E(r(2)(«;)) - A; = ^A;^ + o(fc4) (19) 

y 

and 

Var(T,(s)) = Var(r(^)H) = — fc« + o(A:«). (20) 

lb/ 



For all 77 > 0, write 



The deterministic part in the second-hand right term goes to with k thanks to 
^ ^ Tk(s) - E(Tk(s)) 

( jlQl) . so that we focus on the term ' "^^^ s > 0, because of 

Bienayme-Tchebychev inequality, 

p \nis)-E[n{s)]\ VarjUs)) ^ ^ ( J_ 



This shows the convergence in probability in Lemma 14.101 Moreover, Borel- 
Cantelli Lemma ensures the almost sure convergence as soon as 77 > |. □ 



25 



Cenac, Chauvin, Paccaut, Pouyanne: uncommon suffix tries 



Remark 4.11 Notice that our proof shows actually that the convergence to +00 
in Theorem \4.8\ is valid a.s. (and not only in probability) as soon as (5 > ^. 

4.6 Saturation level for the factorial comb 

In this subsection, we prove Theorem 14.91 

Consider the probabihzed infinite factorial comb defined in Section [2] by 

Vri e N, c„ ^ 



n + l)\ 



The proof hereunder shows actually that ( ^"'"^^"^^ j is an almost surely bounded 

sequence, which implies the result. Recall that TZ denotes the set of all right- 
infinite sequences. By characterization of the saturation level as a function of 
Xn (see ([n])), P(4 ^k) = ¥{3s e 7^, ^ A;) for all positive integers n,k. 

Duality formula ( fT2|) then provides 

F{in^k) = P (3s e 7^, Tkis) ^ n) 

where s' denotes any infinite word having lO'^^^ as a prefix. Markov inequality 
implies 

/ \ •J)'-^^ (x) 

VxG]0,l[, P(4 ^ + ^P(^r(2)(10^-i) <n + A;j ^ ^^J-^ (21) 

(2) 

where ^^^f,_i{x) denotes as above the generating function of the rank of the final 
letter of the second occurrence of lO'^"^ in the infinite random word (f/„ 
The simple form of the factorial comb leads to the explicit expression U{x) 
{i-x)(e=^-i) S'ftsr computation. 



ln>l- 



1 - e^(l - x)) 



"^iS-(^) = ^ • T ' ^" ' ^" -2- (22) 

k\ (e^ - 1) (1 - x) + x^-i (1 - e^(l - x)) 

In particular, applying Formula ( |22l) with n = (A; — 1)! and x = 1 — -(^zrji implies 
that for any k ^ 1, 



fc_i)i^A; + l)< ^ 



1 



, 2k-l 

1 
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Consequently, P [i(k-iy. ^ k + l) — 0{k ^) is the general term of a convergent 
series. Thanks to Borel-Cantelli Lemma, one gets almost surely 

IhS ^^1. 

Let denote the inverse of Euler's Gamma function, defined and increasing on 
the real interval [2, +oo[. If n and k are integers such that {k + l)\ ^ n ^ (A; + 2)!, 
then 

r-i(n) ^ r-i((A; + l)!) ~ /c + 2 ' 
which implies that, almost surely, 

£^ 

n-^oo i [n) 

Inverting Stirling Formula, namely 



when X goes to infinity, leads to the equivalent 

log a; 



hoo log log a; ' 

which implies the result. □ 
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Figure 5: Respective heights and saturation levels for a logarithmic comb (plain 
hues) , a factorial comb (long dashed lines) and a log n-comb (short dashed hues) . 
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